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Abstract Complex macroscopic behaviour can arise in many-body systems with 
only very simple elements as a consequence of the combination of competition and 
inhomogeneity. This paper attempts to illustrate how statistical physics has driven 
this recognition, has contributed new insights and methodologies of wide applica- 
tion influencing many fields of science, and has been stimulated in return. 



1.1 Introduction 

Many body systems of even very simple microscopic constituents with very simple 
interaction rules can show novel emergence in their macroscopic behaviour. When 
the interactions (and any constraints) are also mutually incompatible (frustrated) and 
there is macroscopically relevant quenched disorder, then the emergent macroscopic 
behaviour can be complex (in ways to be discussed) and not simply anticipatable. 
Recent years have seen major advances in understanding such behaviour, in recog- 
nizing conceptual ubiquities across many apparently different systems and in forg- 
ing, transferring and applying new methodologies. Statistical physics has played a 
major part in driving and developing the subject and in providing new methods to 
study and quantify it. This paper is intended to provide a brief broadbrush introduc- 
tion. 

A key part of these developments has been the combination of minimalist 
modelling, development of new concepts and techniques, and fruitful transfers of 
the knowledge between different systems. Here we shall concentrate on a simple 
paradigmic model, demonstrate its ubiquitousness among several often very differ- 
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ent systems, problems and contexts, and introduce some of the useful concepts that 
have arisen. 



1.2 The Dean's Problem and Spin Glasses 

The genesis for the explosion of interest and activity in complexity within the 
physics community was in an attempt to understand a group of magnetic alloys 
known as spin glasses 1 [1]. But here we shall start with a problem that requires no 
physics to appreciate, the Dean's problem [2]. 

A College Dean is faced with the task of distributing N students between two 
dormitories as amicably as possible but given that some pairs of students prefer to 
be in the same dorm while other pairs want to be separated. If any odd number of 
students have an odd number of antagonistic pairwise preferences then their prefer- 
ences cannot all be satisfied simultaneously. This is an example of frustration. The 
Dean's Problem can can be modelled as a mathematical optimization problem by 
defining a cost function H that is to be minimiized: 



where the i,j label students, a = ±1 indicates dorm A/B and the = //;} denote 
the sign and magnitude of the inter-student pair preferences (+ = prefer). We shall 
further concentrate on the situation where the {•/;/} are chosen randomly and in- 
dependently from a single (intensive) distribution P(J) of zero mean 2 , the random 
Dean's Problem. The number of combinations of possible choices grows exponen- 
tially in N (as 2 N ). There is also, in general, no simple local iterative mode of so- 
lution. Hence, in general, when N becomes large the Dean's problem becomes very 
hard, in the language of computer science, NP-complete [3]. 

In fact, the cost function of the random Dean's Problem was already introduced 
in 1975 as a potentially soluble model for a spin glass; there it is known as the 
Sherrington-Kirkpatrick (SK) model [4]. In this model H is the Hamiltonian (or 
energy function), the i, j label spins, the a their orientation (up/down) and the {/} 
are the exchange interactions between pairs of spins. 

In the latter case one was naturally interested in the effects of temperature and 
of phase transitions as it is varied. In the standard procedure of Gibbsian statistical 
mechanics, in thermal equilibrium the probability of a microstate {a} is given by 



H 



(ij) 



= ±1 



(1.1) 




(1.2) 



where Z is the partition function 



Spin glasses were originally observed as magnetic alloys with unusual non-periodic spin order- 
ing. They were also later recognized as having many other fascinating glassy properties 
2 This restriction is not essential but represents the potentially hardest case. 
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Z=J> P [-tf {y} ({ff})/r]; (1.3) 

the subscript {Jn} has been added to H to make explicit that it is for the particular 
instance of the (random) choice of {Jtj}- Again in the spirit of statistical physics one 
may usefully consider typical physical properties over realizations of the quenched 
disorder, obtainable by averaging them over those choices 3 . 

Solving the SK model has been a great challenge and has led to new and subtle 
mathematical techniques and theoretical conceptualizations, backed by new com- 
puter simulational methodologies and experimentation, the detailed discussion of 
which is beyond the scope of this brief report. However a brief sketch will be given 
of some of the conceptual deductions. 

Let us start pictorially. A cartoon of the situation is that of a hierarchically rugged 
landscape to describe the energy/cost as a function of position in the space of micro- 
scopic coordinates and such that for any local perturbations of the microscopic state 
that allow only downhill moves the system rapidly gets stuck and it is impossible to 
iterate to the true minimum or even a state close to it. Adding temperature allows 
also uphill moves with a probability related to exp[— 8H/T] where 8H is the energy 
change. But still for T < T g the system has this glassy hindrance to equilibration, 
a non-ergodicity that shows up, for example, in differences in response functions 
measured with different historical protocols. 

Theoretical studies of the SK model have given this picture substance, clarifi- 
cation and quantification, partly by introduction of new concepts beyond those of 
conventional statistical physics, especially through the work of Giorgio Parisi [1,5]. 

Let us assume that, at any temperature of interest, our system has possibly several 
essentially separate macrostates, which we label by indices {S}. A useful measure 
of similarity of two microstates S,S' is given by their 'overlap', defined as 

Qs*=N~ 1 Y,(Oi)s(Oi)v- (1.4) 

i 

where (o/)s measures the thermal average of a, in macrostate S. 
The distribution of overlaps is given by 

P{J,j}(<l) = I,WsW^(q-q ss ,), (1.5) 

s,s' 

where Ws is the probability of finding the system in macrostate S. 

In general, the macrostates can depend on the specific choice of the {/y} but 
for the SK model the average of P{j..y(q) can be calculated, as also other more 
complicated distributions of the q ss i, such as the correlation of pairwise overlap 
distributions for 3 macrostates S,S',S". 

For a simple (non-complex) system there is only one thermodynamically relevant 
macrostate and hence P(q) has a single delta function peak; at q = for a param- 
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agnet (in the absence of an external field) and at q = m 2 for a ferromagnet, where 
m is the magnetization per spin. In contrast, in a complex system P{q) has structure 
indicating many relevant macrostates 4 . This is the case for the SK model beneath 
a critical temperature and for sufficient frustration, as measured by the ratio of the 
standard deviation of P(J) compared with its mean. Furthemore other measures of 
the ^-distribution indicate a hierarchical structure, ultrametricity and a phylogenic- 
tree structure for relating overlaps of macrostates, chaotic evolution with variations 
of global parameters, and also non-self-averaging of appropriate measures. 

These observations and others give substance to and quantify the rugged land- 
scape picture with macrostate barriers impenetrable on timescales becoming infinite 
with N. For finite-ranged spin glasses this picture must be relaxed to have only finite 
barriers, but still with a non-trivial phase transition to a glassy state. 

The macroscopic dynamics in the spin glass phase also shows novel and inter- 
esting glassy behaviour 5 , never equilibrating and having significant deviations from 
the usual fluctuation-dissipation relationship 6 . 

A brief introduction to the methodolgies to arrive at these conclusions is deferred 
to a later section. 



1.3 Transfers and Extensions 



The knowledge gained from such spin glass studies has been applied to increasing 
understanding of several other physically different systems and problems, via math- 
ematical and conceptual transfers and extensions. Conversely these other systems 
have presented interesting new challenges for statistical physics. In this section we 
shall illustrate this briefly via discussion of some of these transfers and stimulating 
extensions. 

In static/thermodynamic extensions there exist several different analogues of the 
quenched and annealed microscopic variables, {/} and {a} above, and of the inten- 
sive controls, such as T. Naturally, in dynamics of systems with quenched disorder 
the annealed variables (such as the {a} above) become dynamical, but also one can 
consider cases in which the previously quenched parameters are also dynamical but 
with slower fundamental microscopic timescales 7 . 



4 The overline indicates an average over the quenched disorder. 

5 There are several possible microscopic dynamics that leads to the same equilibrium/Gibbsian 
measure, but all such employing local dynamics lead to glassiness. 

6 Instead one finds a modified fluctuation-dissipation relation with the temperature normalized by 
the instantaneous auto-correlation. 

7 Sometimes one speaks of fast and slow microscopic variables but it should be emphasised that 
these refer to the underlying microscopic time-scales. Glassiness leads to much slower macroscopic 
timescales. 
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1.3.1 Optimization and satisfiability 

Already in section (1.2), one example of eqn. (1.1) as an optimization problem was 
given (the Dean's Problem). Another classic hard computer science optimization 
problem is that of equipartitioning a random graph so as to minimise the cross-links. 
In this case the cost function to minimise can be again be written as in eqn. (1.1), 
now with the {/} labelling vertices of the graph, the {Jn} equal to 1 on edges/graph- 
links between vertices and zero where there is no link between i and j, the {<7, = ±} 
indicating whether vertices i are in the first or second partition and with the frus- 
trating constraint a, = imposing equipartitioning. Without the global constraint 
this is a random ferromagnet, but with it the system is in the same complexity class 
as a spin glass. 

Another classic hard optimization problem that extends eqn. (1.1) in an ap- 
parently simple way but in fact leads to new consequence is that of random K- 
satisfiability (/if-SAT) [11]. Here the object is to investigate the simultaneous sat- 
isfiability of many, M, randomly chosen clauses, each made up of K possible mi- 
croscopic conditions involving a large number, N, of binary variables. Labelling the 
variables {<7,} = {±1} and writing jcj to indicate <7, = 1 and jq to indicate o, ■ = — 1, 
a /f-clause has the form 



where the y,-. are xi- or xJJ. In Random K-Sat the {ij} are chosen randomly from the 
N possibilities and the choice of yi . = X(. or xf- is also random, in both cases then 
quenched. In this case one finds, for the thermodynamically relevant typical system, 
that there are two transitions as the ratio a = M/N is increased in the limit N ^ °°; 
for a > OL c \ it is not possible to satify all the clauses simultaneously (UNSAT), for 
a < a c i the problem is satisfiable in principle (SAT), but for a C 2 < oc < a c \ it is very 
difficult to satisfy (in the sense that all simple local variational algorithms stick) and 
this region is known as HARD-SAT. These distinctions are attributable to regions of 
fundamentally different fractionation of the space of satisfiability, different levels of 
complexity. 



1.3.2 K-spin glass 

In fact, again there was a stimulating precursor of this /f-SAT discovery in a "what- 
if extension of the SK model [6, 7] in which the 2-spin interactions of eqn. (1.1) 
are replaced by K-spin interactions: 



(y h ory i2 or....y iK ); i = l,..M 



(1-6) 



h>h—'K a h < - 5 i'2-.-. 



OIK 



(1.7) 
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in which the i 2 ...i K are again chosen randomly and independently from an in- 
tensive distribution of zero mean. In this case, two different phase transitions are 
observed as a function of temperature, a lower thermodynamic transition and a dy- 
namical transition that is at a slightly higher temperature, both to complex spin glass 
phases. The thermodynamic transition represents what is achievable in principle in 
a situation in which all microstates can be accessed; the dynamical transition repre- 
sents the situation where the system gets stuck and cannot explore all the possibili- 
ties, analogues of HARDS AT-UNS AT and SAT-HARDS AT. 

The /f-spin glass is also complex with a non-trivial overlap distribution function 
P{q) but now the state first reached as the transitions are crossed has a different 
structure from that found for the 2-spin case. Now 

P(q) = (1 -x)8(q-q min )+x8(q-q mia ); (1.8) 

in contrast with the SK case where there is continuous weight below the maximum 
qmax- The two delta functions demonstrate that there is still the complexity of many 
equivalent but different macrostates, but now with equal mutual orthogonalities (as 
compared with the 2-spin SK case where there is a continuous range of macrostates). 
This situation turns out to be quite common in many extensions beyond SK. 



1.3.3 Statics, dynamics and temperature 

At this point it is perhaps useful to say a few more words about the differences be- 
tween statics/thermodynamics and dynamics in statistical physics, and about types 
of micro-dynamics and analogues of temperature. 

In a physical system one often wishes to study thermodynamic equilibrium, as- 
suming all microstates are attainable if one waits long enough. In optimization prob- 
lems one typically has two types of problem; the first determining what is attainable 
in principle, the second considering how to attain it. The former is the analogue of 
thermodynamic equilibrium, the latter of dynamics. 

In a physical system the true microscopic dynamics is given by nature. However, 
in optimization studies the investigator has the opportunity to determine the micro- 
dynamics through the computer algorithms he or she chooses to employ . 

Temperature enters the statistical mechanics of a physical problem in the stan- 
dard Boltzmann-Gibbs ensemble fashion, or as a measure of the stochastic noise in 
the dynamics. We have already noted that it can also enter an optimization prob- 
lem in a very similar fashion if there is inbuilt uncertainty in the quantity to be 
optimized. But stochadtic noise can also usefully be introduced into the artificial 
computer algorithmic dynamics used to try to find that optimum. This is the basis 
of the optimization technique of simulated annealing where noise of variance Ta 
is deliberately introduced to enable the probabalistic scaling of barriers, and then 
gradually reduced to zero [8]. 
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1.3.4 Neural networks 

The brain is made up of a very large number of neurons, firing at different rates and 
extents, interconnected by an even much larger number of synapses, both excitory 
and inhibitory. In a simple model due to Hopfield [9] one can consider a cartoon 
describable again by a control function of the form of eqn. (1.1). In this model 
the neurons {/} are idealized by binary McCullough-Pitts variables {(J; = ±1}, the 
synapses by {Jij}, positive for excitatory and negative for inhibitory, with stochastic 
neural microdynamics of effective temperature r neura i emulating the width of the 
sigmoidal response of a neuron's output to the combined input from all its afferent 
synapses, weighted by the corresponding activity of the afferent neurons. 

The synapses are distributed over both signs, yielding frustration and apparently 
random at first sight. However actually they are coded to enable attractor basins 
related to memorized patterns of the neural microstates {§,■}; fJ. = l,...p = OcN. 
The similarity of a neural micro-state to a pattern ji is given by an overlap 

m^=N- l ^i)s^. (1.9) 

i 

Retrieval of memory ji is the attractor process in which a system started with a small 
iterates towards a large value of m* 1 . 

In Hopfield's original model he took the {Jij} to be given by the Hebb-inspired 
form 

with randomly quenched {£,-} 8 . For a less than a T neura i -dependent critical value 
(X c {T neura i) patterns can be retrieved. Beyond it only quasi-random spin glass min- 
ima unrelated to the memorised patterns remain ( and still only for T neura i not too 
large). However, other {Jij} permit a slightly larger capacity (as also can occur for 
correlated patterns). 

Again the landscape cartoon is illustratively useful. It can be envisaged as one 
for ffr/j.} as a function of the neural microstates (of all the neurons), with the dy- 
namics one of motion in that landscape, searching for minima using local deviation 
attempts. The memory basins are large minima. Clearly one would like to have 
many different retrievable memories. Hence frustration is necessary. But equally, 
too much frustration would lead to a spin-glass-like state with minima unrelated to 
learned memories. 

This cartoon also leads immediately to the recognition that learning involves 
modifying the landscape so as to place the attractor minima around the states to 
be retrieved. This extension can be modelled minimally via a system of coupled 
dynamics of neurons whose state dynamics is fast (attempting retrieval or general- 
ization) and synapses that also vary dynamically but on a much slower timescale 
and in response to external perturbations (yielding learning) [10]. 
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More examples of many-body systems with complex macrobehaviour are to be 
found in social systems, in which the microscopic units are people (or groups of 
people or institutions), sometimes co-operating, often competing. Here explicit dis- 
cussion will be restricted to one simple model, the Minority Game [12], devised to 
emulate some features of a stockmarket. N 'agents' play a game in which at each 
time-step each agent makes one of two choices with the objective to make the choice 
which is in the minority 9 . They have no direct knowledge of one another but (in the 
original version) make their choices based on the commonly-available knowledge 
of the historical actual minority choices, using their own individual stategies and 
experience to make their own decisions. In the spirit of minimalism we consider 
all agents (i) to have the same 'memories', of the minority choices for the last m 
time-steps, (ii) to each have two strategies given by randomly chosen and quenched 
Boolean operators that, acting on the m-string of binary entries representing the mi- 
nority choices for the last m steps, output a binary instruction on the choice to make, 
(iii) using a personal 'point-score' to keep tally of how their strategies would have 
performed if used, increasing the score each time they would have chosen the actual 
minority, and (iv) using their strategy with the larger point-score. Frustration is rep- 
resented in the minority requirement, while quenched disorder arises in the random 
choice of individual strategies. 

Simulational studies of the 'volatility', the standard deviation of the actual minor- 
ity choice, shows (i) a deviation from individually random choices, indicating cor- 
relation through the common information, (ii) a cusp-minimum at a critical value a t 
of the ratio of the information dimension to the number of agents a=D /N = 2 m /N, 
suggesting a phase transition at a c , (iii) ergodicity for a > a c but non-ergodic de- 
pendence on the point-score initialization for a < a c , indicating that the transition 
represents the onset of complexity. This is reminiscent of the cusp and the ergodic- 
nonergodic transition observed in the susceptibilities of spin glass systems as the 
temperature is reduced through the spin glass transition. 

Furthermore, this behaviour is essentially unaltered if the 'true' history is re- 
placed by a fictitious 'random' history at each step, with all agents being given the 
same false history, indicating that it principally represents a carrier for an effective 
interaction between the agents. Indeed, generalising to a D-dimensional random 
history information space, considering this as a vector-space and the strategies as 
quenched D- vectors of components {R'] >1 }; s = 1,2, fl = 1, ..D in that space, and 
averaging over the stochastically random 'information', one is led to an effective 
control function analagous to those of eqn. (1.1) and eqn. (1.10) with p replaced 
by a, now {if;,- = (R\ — Rf)/2}, an extra multiplicative minus sign on the right 
hand side of eqn. (1.10), and also a random-field term dependent upon the {fy} 
and {©, = (Rf +Rj)/2}. As noted, there is an ergodic-nonergodic transition at a cr- 



The philosophy is that one gets the best price by selling when most want to buy or buying when 
most want to sell. 
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titical a, but now the picture is one of the {^, } as repellers rather than the attractors 
of the Hopfield model 10 . 

The typical behaviour of this system, as for the spin glasses, can be studied us- 
ing a dynamical generating functional method [13], averaged over the choice of 
quenched strategies, in a manner outlined below. The averaged many-body system 
can then be mapped into an effective representative agent ensemble with memory 
and coloured noise, with both the noise correlations and the memory kernel deter- 
mined self-consistently over the ensemble. Note that this is in contrast to (and cor- 
rects) the common assumption of a single deterministic representative agent. The 
phase transition from ergodic to non-ergodic is manifest by a singularity in the two- 
time point-sign correlation function 

C(t,t') =Ar 1 £sgn(p i (t)sgn(p i (t') = <sgn(p(t)sgn(p(t')) c „ s (1.11) 

i 

where the first equality refers to the many-body problem and the second its equiva- 
lence in the effective agent ensemble. 



1.4 Methodologies 

For systems in equilibrium, physical observables are given by InZ evaluated for 
the specific instance of any quenched parameters, or strictly the generalized gen- 
erating function lnZ({A}) where the {X} are generating fields to be taken to zero 
after an appropriate operation (such as d/d A) is performed. Hence the average over 
quenched disorder is given by InZ. One would like to perform the average over 
quenched disorder explicitly to yield an effective system. However, since Z is a sum 
over exponentials of a function of the variables, InZ is difficult to average directly 
so instead one uses the relation 

\nZ = Lim n ^ Q n- l {Z n - 1) (1.12) 

and interprets the Z" as corresponding to a system whose variables have extra 
'replica' labels, a = l,..n, for which one can then average the partition function, 
an easier operation, at the price of needing to take the eventual limit n — > 0. The 
relevant 'order parameters' are then correlations between replicas 

q"P=N- 11 E(o?of)T (1.13) 



10 One can make the model even more minimal by allowing each agent only one strategy {^,} 
which (s)he either follows if its point-score is positive or acts oppositely to if the point-score is 
negative. This removes the random-field term and also the cusp in the tabula rasa volatility, but 
retains the ergodic-nonergodic transition[14]. 
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where refers to a thermal average in the effective post-averaging system. This 
order parameter is non-zero in the presence of frozen order, but more interestingly 
(and subtly) also exhibits the further remarkable feature of spontaneous replica sym- 
metry breaking, indicating complexity. After further subtleties beyond the scope of 
this short introduction, there emerges an order function q(x);x £ [0.1] from which 
the average overlap function is obtained by 

P{qj = dx/dq (1.14) 

For dynamics the analogue of the partition function Z is a generating functional, 
which may be written symbolically as 

Zdyn = / 5 (microscopic eqns . of motion) exp ( { A } ) (1-15) 

all variables, all times 

where the symbolize the microscopic variables and a Jacobian is implicit. Av- 
eraging over the quenched disorder now induces interaction between epochs and 
integrating out the microscopic variables results in the effective single agent en- 
semble formulation, as well as emergent correlation and response functions as the 
dynamic order parameter analogues of the static inter-replica overlaps, exhibiting 
non-analyticity at a phase transition to non-ergodicity. 



1.5 Conclusion 

A brief illustration has been presented of how complex co-operative behaviour arises 
in many body systems due to the combination of frustration and disorder in the mi- 
croscopies of even very simply formulated problems with very few parameters. Such 
systems are not only examples of Anderson's famous quotation "More is different" 
but also demonstrate that frustration and disorder in microscopies can lead to com- 
plexity in macroscopics; i.e. many and complexly related different?,. Furthermore, 
this complexity arises in systems with very simple few-valued microscopic param- 
eters; complexity is not the same as complication and does not require it. 

There has also been demonstrated valuable transfers between systems that appear 
very different at first sight, through the media of mathematical modelling, conceptu- 
alization and investigatory methodologies, a situation reminiscent of the successful 
use of the Rosetta stone in learning an unknown language script by comparison with 
another that carries the same message in a different format. 

The perspective taken has been of statistical physics, but it must be emphasised 
that the stimulation has been both from and to physics, since many of these com- 
plex systems have interesting features in their microscopic underpinning that are 
richer than those in the physics of conventional dictionary definition and provide 
new challenges to the physicist. 

Also of note is how a blue skies attempt to understand some obscure magnetic 
alloys through soluble but, for the experimental alloys, unphysical modelling has 
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led to an explosion of appreciation of new concepts, understanding and application 
of ideas and methologies throughout an extremely wide range of the sciences. 
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